Arize Phoenix
AI Observability and Evaluation
Last updated
Was this helpful?
AI Observability and Evaluation
Last updated
Was this helpful?
Phoenix is an open-source observability tool designed for experimentation, evaluation, and troubleshooting of AI and LLM applications. It allows AI engineers and data scientists to quickly visualize their data, evaluate performance, track down issues, and export data to improve. Phoenix is built by , the company behind the industry-leading AI observability platform, and a set of core contributors.
Phoenix works with OpenTelemetry and instrumentation. See Integrations: Tracing for details.
Phoenix offers tools to workflow.
- Create, store, modify, and deploy prompts for interacting with LLMs
- Play with prompts, models, invocation parameters and track your progress via tracing and experiments
- Replay the invocation of an LLM. Whether it's an LLM step in an LLM workflow or a router query, you can step into the LLM invocation and see if any modifications to the invocation would have yielded a better outcome.
- Phoenix offers client SDKs to keep your prompts in sync across different applications and environments.
Running Phoenix for the first time? Select a quickstart below.
Check out a comprehensive list of example notebooks for LLM Traces, Evals, RAG Analysis, and more.
Add instrumentation for popular packages and libraries such as OpenAI, LangGraph, Vercel AI SDK and more.
Join the Phoenix Slack community to ask questions, share findings, provide feedback, and connect with other developers.
is a helpful tool for understanding how your LLM application works. Phoenix's open-source library offers comprehensive tracing capabilities that are not tied to any specific LLM vendor or framework.
Phoenix accepts traces over the OpenTelemetry protocol (OTLP) and supports first-class instrumentation for a variety of frameworks (, ,), SDKs (, , , ), and Languages. (, , etc.)
Phoenix is built to help you and understand their true performance. To accomplish this, Phoenix includes:
A standalone library to on your own datasets. This can be used either with the Phoenix library, or independently over your own data.
into the Phoenix dashboard. Phoenix is built to be agnostic, and so these evals can be generated using Phoenix's library, or an external library like , , or .
to attach human ground truth labels to your data in Phoenix.
let you test different versions of your application, store relevant traces for evaluation and analysis, and build robust evaluations into your development process.
to test and compare different iterations of your application
, or directly upload Datasets from code / CSV
, export them in fine-tuning format, or attach them to an Experiment.
